exploit social context review qualiti predict homophili digit world livejourn case studi answer web queri structur data sourc releas search queri click privat search person queri page topic analysi prune polici dtier invert index correct guarante web search engin maintain larg scale invert index queri thousand time user eager inform order cope vast amount queri load search engin prune index document return top result prune index comput batch result approach improv perform reduc size index comput top result prune index notic signific degrad result qualiti document top result includ prune index result comput prune index fierc competit onlin search market phenomenon undesir paper studi avoid degrad result qualiti due prune base perform optim realiz benefit contribut number modif prune techniqu creat prune index result comput algorithm guarante top match page top search result comput batch prune index time show determin optim size prune index experiment evalu algorithm collect million web page answer bound continu search queri world wide web search queri appli extract relev inform world wide web period time denot continu search queri improv continu search queri concern qualiti retriev result fresh result time avail respect data object web notif user search engin case user notifi immedi respect inform decreas quickli new compani affect respect stock sale offer product longer short period time document filter literatur optim queri base threshold classif document qualiti threshold return user threshold tune order optim qualiti retriev result disadvantag classifi web data directori structur model manag text databas classifi web data directori structur detect spam web page content analysi paper continu investig web spam inject artifici creat page web order influenc result search engin drive traffic page fun profit paper consid previous undescrib techniqu automat detect spam page examin effect techniqu isol aggreg classif algorithm combin heurist correctli identifi spam page judg collect page misidentifi spam spam page model manag content text databas larg amount valuabl inform store web access text databas metasearch provid unifi interfac queri multipl databas effici metasearch reli succinct statist summari databas content select databas queri databas select research larg assum databas static statist summari chang time databas rare static statist summari describ content updat period reflect content paper report result studi show content summari real web databas evolv period week show surviv analysi techniqu gener cox proport hazard regress model databas time predict updat content summari final exploit chang model devis updat schedul summari date contact databas need evalu qualiti schedul experiment real web databas download textual hidden web content keyword queri increas amount inform web todai search interfac user type set keyword search form order access page web site page refer hidden web deep web static link hidden web page search engin discov index page return result recent studi content provid hidden web site high qualiti extrem valuabl user paper studi build effect hidden web crawler autonom discov download page hidden web entri point hidden web site queri interfac main challeng hidden web crawler face automat gener meaning queri issu site provid theoret framework investig queri gener problem hidden web propos effect polici gener queri automat polici proce iter issu queri iter experiment evalu effect polici real hidden web site result promis instanc experi polici download hidden web site million document issu fewer queri web directori construct lexic chain abstract web directori provid locat relev inform web typic web directori reli human put signific time effort find import page web categor directori paper present autom creation web directori high level method take input subject hierarchi collect page leverag varieti lexic resourc natur languag process commun enrich hierarchi process page identifi sequenc import term refer lexic chain final lexic chain order decid enrich subject hierarchi assign page experiment result real web data show method promis assist human page categor directoryrank order page web directori web directori repositori web page organ hierarchi topic topic paper present directoryrank rank framework order page topic inform topic method work step process web page topic order extract structur call lexic chain measur inform page topic measur rel semant similar page topic final metric combin rank page topic present user infoci web search engin improv web search linguist analysi paper present infoci web search engin goal creat infoci improv peopl find inform web resolv ambigu present natur languag text achiev perform linguist analysi content web page index departur exist web search engin return result base keyword match addit step linguist process infoci main advantag infoci gain deeper understand content web page match user queri index document improv relev return result base linguist process infoci organ present result user intuit wai paper present linguist process technolog incorpor infoci appli help user find inform web effici discuss compon architectur infoci benefit ad linguist process final experiment evalu perform compon leverag linguist inform order categor web page infoci web search engin improv web search linguist analysi paper present infoci web search engin goal creat infoci improv peopl find inform web resolv ambigu present natur languag text achiev perform linguist analysi content web page index departur exist web search engin return result base keyword match addit step linguist process infoci main advantag infoci gain deeper understand content web page match user queri index document improv relev return result base linguist process infoci organ present result user intuit wai paper present linguist process technolog incorpor infoci appli help user find inform web effici discuss compon architectur infoci benefit ad linguist process final experiment evalu perform compon leverag linguist inform order categor web page web evolut web search engin perspect seek gain improv insight web search engin cope evolv web attempt provid user date result purpos collect weekli snapshot web site year measur evolut content link structur measur focu aspect potenti interest search engin design evolut link structur time rate creation page distinct content web rate chang content exist page search centric measur degre chang effect chang detect sampl larg scale data intens environ world wide web data wareh make local copi remot data sourc due limit network comput resourc difficult monitor sourc constantli check download chang data item copi scenario goal detect fix download resourc paper propos sampl base download polici identifi chang data item effect sampl base approach sampl small number data item data sourc download data item sourc chang sampl analyz effect sampl base polici compar propos polici exist includ state art frequenc base polici experi synthet real world data show rel merit polici great potenti sampl base polici case sampl base polici download chang item exist polici www search engin evalu normal perform highli inflect languag morphosyntact lexicon basi implement greek wordnet 